Skip to content

Conversation

@LuciferYang
Copy link
Contributor

@LuciferYang LuciferYang commented Jan 27, 2023

What changes were proposed in this pull request?

This PR aims to add ExtendedConnectTest to disable tests related to RemoteSparkSession selectively.

Why are the changes needed?

RemoteSparkSession related tests need to be prepared shaded connect-server.jar, so this causes developers must to run sbt package or maven clean install -DskipTests to prepare shaded connect-server.jar before testing connect-client-jvm, it seems a little cumbersome, so this pr add a new test tag ExtendedConnectTest to make developers have the ability to disable relevant test cases through -Dtest.exclude.tags=org.apache.spark.tags.ExtendedConnectTest.

Does this PR introduce any user-facing change?

No

How was this patch tested?

  • Pass GitHub Actions
  • Manual test

SBT

build/sbt clean "connect-client-jvm/test" -Dtest.exclude.tags=org.apache.spark.tags.ExtendedConnectTest
[info] Run completed in 2 seconds, 416 milliseconds.
[info] Total number of tests run: 24
[info] Suites: completed 4, aborted 0
[info] Tests: succeeded 24, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.

All tests passed and ClientE2ETestSuite skipped

Maven

build/mvn clean install -pl connector/connect/client/jvm -am -DskipTests
build/mvn clean test -pl connector/connect/client/jvm -Dtest.exclude.tags=org.apache.spark.tags.ExtendedConnectTest
Run completed in 964 milliseconds.
Total number of tests run: 24
Suites: completed 5, aborted 0
Tests: succeeded 24, failed 0, canceled 0, ignored 0, pending 0
All tests passed.

All tests passed and ClientE2ETestSuite skipped

@LuciferYang LuciferYang changed the title [SPARK-42209][CORE][CONNECT] Introduce test tag ExtendedConnectTest [SPARK-42209][CORE][CONNECT] Introduce ExtendedConnectTest to make user can disable tests related to RemoteSparkSession selectively Jan 27, 2023
@LuciferYang LuciferYang changed the title [SPARK-42209][CORE][CONNECT] Introduce ExtendedConnectTest to make user can disable tests related to RemoteSparkSession selectively [SPARK-42209][CORE][CONNECT] Introduce ExtendedConnectTest to make user can disable tests related to RemoteSparkSession selectively Jan 27, 2023
@LuciferYang LuciferYang marked this pull request as draft January 27, 2023 09:49
@LuciferYang LuciferYang marked this pull request as ready for review January 27, 2023 10:11
@LuciferYang LuciferYang marked this pull request as draft January 27, 2023 10:56
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the repl dependency fixes the issue, I'd like to avoid this addition, @LuciferYang .

@LuciferYang
Copy link
Contributor Author

If the repl dependency fixes the issue, I'd like to avoid this addition, @LuciferYang .

OK, I will close this one

@dongjoon-hyun
Copy link
Member

dongjoon-hyun commented Jan 27, 2023

Thank you for investigating all issues and alternatives, @LuciferYang !

@LuciferYang
Copy link
Contributor Author

Thanks for your help and correction @dongjoon-hyun

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants